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June  21,  1996 

MEMORANDUM  FOR  DIRECTOR,  DEFENSE  INFORMATION  SYSTEMS 

AGENCY 

SUBJECT:  Audit  Report  on  Certification  and  Management  of  Value-Added  Networks 
(Report  No.  96-172) 


We  are  providing  this  audit  report  for  your  review  and  comment.  Management 
comments  on  a  draft  of  this  report  were  considered  in  preparing  the  final  report. 


DoD  Directive  7650.3  requires  that  all  recommendations  and  potential  monetary 
benefits  be  resolved  promptly.  Management  nonconcurred  with  Recommendation  1. 
We  request  that  the  Defense  Information  Systems  Agency  reconsider  its  position  and 
provide  additional  comments  on  Recommendation  1.  in  response  to  this  final  report. 
Comments  must  be  received  by  July  22,  1996. 


We  appreciate  die  courtesies  extended  to  the  audit  staff.  Questions  on  the  audit 
should  be  directed  to  Ms.  Kimberley  A.  Caprio,  Audit  Program  Director,  at 
(703)  60^9248  (DSN  664-9248)  or  Ms.  Carolyn  R.  Davis,  Audit  Project  Manager,  at 
(703)  604-9217  (DSN  664-9217).  See  Appendix  F  for  the  report  distribution.  The 
audit  team  members  are  listed  inside  the  back  cover. 


Robert  J.  Lieberman 
Assistant  Inspector  General 
for  Auditing 


Office  of  the  Inspector  General,  DoD 

Report  No.  96-172  June  21, 1996 

(Project  No.  6CA-(X)19) 

Certification  and  Management  of  Value-Added  Networks 


Executive  Summary 


Introduction.  Value-Added  Networks  provide  communication  of  electronic  data 
between  DoD  and  its  trading  partners,  ^ch  Value-Added  Network  must  be  certified 
by  the  Government.  Twenty-five  Value-Added  Networks  had  been  certified  and 
fourteen  Value-Added  Networks  were  awaiting  certification  as  of  January  1996.  The 
Defense  Information  Systems  Agency  is  responsible  for  certifying  Value-Added 
Networks  for  all  Government  organizations,  including  DoD. 

Audit  Objectives.  The  overall  audit  objective  was  to  determine  the  adequacy  of  the 
Value-Added  Network  certification  process  and  of  the  management  and  oversight  of 
Value-Added  Networks.  We  also  reviewed  the  management  control  programs  as  they 
applied  to  the  overall  audit  objective  at  the  DoD  organizations  we  visited. 

Audit  Results.  The  Defense  Information  Systems  Agency  did  not  establish  an 
adequate  Government  Value-Added  Network  certification  process  and  did  not 
adequately  nionitor  Value-Added  Networks  for  compliance  with  the  Value-Added 
Network  License  Agreement.  As  a  result,  15  of  the  25  Value-Added  Networks  were 
certified  even  though  the  adequacy  of  their  financial  resources  was  questionable,  and 
die  Government  cannot  ensure  that  control  is  exercised  to  prevent,  identify,  and  resolve 
deficient  services  by  the  Value-Added  Networks.  The  Government  and  its  trading 
partners  may  be  impacted  by  the  potential  loss  of  business.  See  Part  I  for  a  discussion 
of  die  toding. 

Management  controls  over  Value-Added  Network  certification,  management,  and 
oversight  needed  improvement.  See  Appendix  A  for  details  on  our  review  of  the 
management  control  program.  Recommendations  in  this  report,  if  implemented,  will 
bring  about  improvements  in  the  certification  of  Value-Added  Networks  and 
ensure  Value-Added  Network  compliance  with  the  Value-Added  Network  License 
Agreement. 

Summaity  of  Recommendations.  We  recommend  that  the  Director,  Defense 
Information  Systems  Agency: 

0  issue  policy  requiring  enforcement  of  compliance  with  the  Federal 
Acquisition  Regulation  9.104,  "Contractor  Qualifications,"  to  include  establishing  a 
system  for  evaluating  business  qualifications  such  as  a  weighted  procedure  or  point 
system; 


o  issue  policy  for  monitoring  Value-Added  Networks  for  compliance  with  the 
Value-Added  Network  License  Agreement;  and 

o  expedite  the  completion  and  issuance  of  the  new  Value-Added  Network 
License  Agreement. 


Management  Comments.  The  Defense  Information  Systems  Agency  partially 
concurred  with  the  draft  report  recommendations.  The  comments  stated  that  the 
Defense  Information  Systems  Agency  either  has  implemented  or  plans  to  implement 
each  of  the  recommendations.  T^e  comments  also  stated  that  the  Defense  Information 
Systems  Agency  currently  has  procedures  regarding  contractor  responsibility  that  are  in 
accordance  with  the  Federal  Acquisition  Regulation  requirements  and,  therefore,  it 
does  not  see  a  need  to  revise  current  procedures  to  determine  contractor  responsibility 
in  accordance  with  the  Federal  Acquisition  Regulation.  See  Part  I  for  a  summary  of 
management  comments  and  Part  HI  for  the  complete  text  of  management  comments. 

Audit  Response.  Management's  comments  were  partially  responsive.  Although 
management  does  not  see  a  need  for  revising  current  procedures  to  determine 
contractor  responsibility,  we  maintain  that  there  is  a  need  for  a  more  objective 
evaluation  to  ensure  diat  each  Value-Added  Network  meets  minimum  standards  and 
that  Value-Added  Networks  being  certified  are  financially  capable  and  will  be  in 
business  for  the  foreseeable  future.  We  request  that  the  Defense  Information 
Systems  Agency  reconsider  its  position  and  provide  additional  conmients  on  the 
recommendation  by  July  22,  1996. 


11 


Table  of  Contents 


Executive  Summary  i 

Part  I  -  Audit  Results 

Audit  Background  2 

Audit  Objectives  3 

Value-Added  Network  Certification,  Management,  and  Oversight  4 

Part  n  -  Additional  Information 

Appendix  A.  Scope  and  Methodology 
Value-Added  Network  Certification,  Management,  and  Oversight 
Review  14 

Audit  Period,  Standards,  and  Locations  14 

Management  Control  Program  15 

Appendix  B.  Summary  of  Prior  Audits  and  Other  Reviews  16 

Appendix  C.  Value-Added  Network  License  Agreement 

Requirements  17 

Appendix  D.  Organizations  Visited  or  Contacted  18 

Appendix  E.  Report  Distribution  20 

Part  in  -  Management  Conunents 

Defense  Information  Systems  Agency  Comments  24 


Part  I  -  Audit  Results 


Audit  Results 


Audit  Background 

This  audit  was  conducted  as  a  result  of  work  performed  on  Inspector  General, 
DoD,  Project  No.  5CA-3002,  "Audit  of  DoD  Implementation  of  Electronic 
Commerce  in  Contracting  for  Small  Purchases."  The  review  identified  potential 
systemic  problems  wiA  the  implementation  of  the  Federal  Acquisition 
Computer  Network  (FACNET).  FACNET  is  the  network  through  which  the 
Government  communicates  transactions  electronically  between  its  individual 
procurement  offices  and  its  trading  partners.  The  potential  systemic  problems 
included  concerns  about  the  adequacy  of  the  certification  process  for  Value- 
Added  Networks  (VANs). 

VANs  are  an  integral  part  of  the  current  and  proposed  FACNET.  A  VAN  is  a 
commercial  company  that  provides  communication  of  electronic  data 
interchange  (EDI)  between  DoD  and  trading  partners.  A  Value-Added  Service 
(VAS)  is  a  commercial  company  or  VAN  that  provides  extra-fee-based  services 
such  as  "EDI-to-fax"  services,  complete  EDI-integrated  business  systems,  and 
translation  services,  in  addition  to  providing  communications  capabilities. 

The  figure  shows  the  role  of  the  VAN  between  DoD  and  the  trading  partner  and 
the  various  services  offered  by  the  VAS. 


Source:  Office  of  the  Deputy  Under  Secretary  of  Defense  (Acquisition  Reform). 
Value-Added  Networks  and  Value-Added  Services 
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Audit  Results 


Each  VAN  must  be  certified  by  the  Government.  The  VAN  certification 
process  should  include: 

o  the  signing  of  the  VAN  License  Agreement  (VLA)  by  the  prospective 

VAN; 


0  a  business  qualification  review  in  accordance  with  Federal  Acquisition 
Regulation  (FAR)  part  9,  "Contractor  Qualifications;" 

o  successful  completion  of  communications  testing;  and 

0  the  signing  of  the  VLA  by  the  Defense  Information  Technology 
Contracting  Office  (DITCO),  an  office  within  the  Defense  Information  Systems 
Agency  (DISA),  upon  completion  of  the  previously  mentioned  steps. 

The  VLA  is  a  no-cost  agreement  between  a  VAN  and  the  Government  allowing 
the  VAN  access  to  electronic  commerce  data  and  allowing  the  Government  to 
use  the  data  interchange  capability  and  VAN  services. 

DISA  is  responsible  for  certifying,  managing,  and  overseeing  VANs  for  all 
Government  organizations,  including  DoD.  As  of  January  1996,  25  certified 
VANs  existed  with  14  awaiting  certification.  Of  the  25  certified  VANs,  13  are 
also  VAS  companies  that  translate  data. 


Audit  Objectives 


The  overall  audit  objective  was  to  determine  the  adequacy  of  the  VAN 
certification  process  and  of  the  management  and  oversight  of  VANs.  We  also 
reviewed  the  management  control  program  at  each  organization  visited  as  it 
applied  to  the  over^l  audit  objective.  See  the  finding  for  a  discussion  of  the 
management  control  weaknesses  we  identified  and  Appendix  A  for  the  audit 
scope  and  methodology  and  details  of  our  review  of  the  management  control 
programs.  See  Appendix  B  for  a  summary  of  prior  coverage  related  to  the 
overall  audit  objective. 
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Value-Added  Network  Certification, 
Management,  and  Oversight 

The  Defense  Information  Systems  Agency  had  not  established  an 
adequate  process  for  certifying  Value-Added  Networks.  DISA  also  was 
not  adequately  monitoring  Value-Added  Networks  to  verify  compliance 
with  VAN  License  Agreement  provisions.  Tliose  conditions  occurred 
because  DISA  management  wanted  to  encourage  participation  in 
FACNET  so  they  chose  to  be  lenient  in  certifying  V^s  and  did  not 
always  comply  with  existing  guidelines,  including  the  FAR  part  9  and 
the  VLA.  As  a  result,  the  Government,  including  DoD,  cannot  ensure 
that  VANs  being  relied  upon  to  perform  VAN  services  are  financially 
responsible  and  that  adequate  and  continuous  control  is  being  exercised 
to  prevent,  identify,  and  resolve  deficient  VAN  services.  Consequently, 
the  Government  and  trading  partners  may  be  impacted  by  the  potential 
loss  of  business. 


Background 


The  FAR  part  9,  "Contractor  Qualifications,"  states  that  all  prospective 
contractors  should  be  responsible.  It  further  states  that  to  be  determined 
responsible,  a  prospective  contractor  must  have  adequate  financial  resources  to 
perform  the  contract  or  the  ability  to  obtain  them.  FAR  part  9  also  states  that  a 
prospective  contractor  must  have  the  necessary  organization,  experience, 
accounting  and  operational  controls,  and  technical  skills,  or  the  ability  to  obtain 
them,  to  &  determined  responsible. 

DITCO  evaluates  a  candidate  for  VAN  certification  by  evaluating  criteria  such 
as  customer  references,  financial  institution  references,  its  Dun  &  Bradstreet 
rating,  and  its  operational  business  plan.  DITCO  also  makes  sure  that  the 
prospective  VAN  is  not  on  the  vendor  debarment  list. 

The  VAN  License  Agreement  DCA  200-94-H-0015  establishes  the  terms  and 
conditions  for  doing  business  electronically  with  the  Government.  The  VLA 
requirements  are  outlined  in  Appendix  C. 


Adequacy  of  the  VAN  Certification  Process 


Application  of  FAR  Part  9.  To  encourage  participation  in  FACNET,  when 
certifying  VANs,  DISA  chose  not  to  strictly  comply  with  the  policies  prescribed 
in  FAR  part  9  regarding  contractor  financial  qualifications.  DITCO  chose  not 
to  perform  further  evaluations  to  identify  payment  history  and  current  financial 
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status  on  prospective  VANs,  despite  recommendations  made  by 
Dun  &  Bradstreet,  before  certifying  those  candidates  as  VANs.  Also,  DITCO 
approved  organizations  with  questionable  financial  resources  as  VANS. 

No  Dun  &  Bradstreet  Ratings.  DITCO  did  not  perform  additional 
research  on  prospective  VANs  that  Dun  &  Bradstreet  could  not  rate-those  it 
identified  as  having  questionable  financial  capabilities  and  needing  additional 
support  prior  to  certification.  To  determine  whether  a  prospective  VAN  was 
financially  responsible  in  accordance  with  FAR  part  9,  DITCO  relied  on  reports 
it  requested  from  Dun  &  Bradstreet.  Dun  &  Bradstreet  performs  financial  risk 
assessments  of  companies  and  generates  reports  that  show  the  resources  of  the 
company.  Dim  &  Bradstreet  assessments  are  based  on  a  review  of  the  payment 
history  and  the  current  financial  statements  of  the  company.  If  Dun  & 
Bradstreet  does  not  have  access  to  a  sufficient  sample  of  payment  experiences  or 
a  current  financial  statement.  Dun  &  Bradstreet  provides  no  rating,  but  rather 
recommends  to  the  Defense  Lxigistics  Agency  that  a  more  detailed  evaluation  be 
done. 


Use  of  Dun  &  Bradstreet  Ratings  and  Other  Financial  Data.  DITCO 
personnel  responsible  for  performing  evaluations  were  lenient  in  approving 
VANs  even  when  the  adequacy  of  financial  resources  was  questionable.  Our 
review  of  the  evaluations  performed  on  the  25  certified  VANs,  as  of 
October  10,  1995,  determined  that  DITCO  certified  15  of  the  25  VANS  despite 
the  lack  of  Dim  &  Bradstreet  ratings  or  moderate  Dun  &  Bradstreet  ratings 
suggesting  close  examination  of  the  company.  It  also  approved  candidates  with 
bank  balances  below  $10,000. 

The  table  shows  the  evaluation  status  of  the  15  certified  VANs  with 
questionable  financial  capabilities. 


Evaluation  Status  of  VANs  With  Questionable  Financial  Capabilities 

Evaluation  Status 

Number  of  VANs* 

Dun  &  Bradstreet  recommended  for  further  evaluation 

10 

No  Dun  &  Bradstreet  ratings 

8 

Moderate  Dun  &  Bradstreet  Rating 

5 

Bank  balance  below  $10,000 

2 

*Some  VANs  were  deficient  in  more  than  one  area. 

For  example,  a  VAN 

may  have  needed  further  evaluation  and  also  have  had  a  moderate  Dun  & 

Bradstreet  rating. 

Rationale  for  Limited  Evaluations.  According  to  DITCO  personnel,  further 
evaluations  were  not  performed  and  more  stringent  requirements  were  not 
applied  because  the  VLA  was  considered  a  "no  cost"  agreement.  According  to 
DISA  management,  because  the  contract  is  a  "no  cost"  agreement,  the  dollar 
limitation  of  the  VLA  is  below  the  simplified  acquisition  tlueshold  of  $100,000 
and,  therefore,  is  not  worth  the  cost/benefit  analysis.  That  is,  the  VANs 
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charged  only  ihe  trading  partners  for  FACNET  services  performed  by  the  VAN, 
and  the  VIA  provided  for  the  VANs  to  make  their  services  available  to 
Government  agencies,  including  DoD,  at  no  cost.  However,  DITCO  should 
have  performed  further  evaluations  as  recommended  by  EHm  &  Bradstreet  to 
provide  assurance  of  VAN  credibility,  regardless  of  the  simplified  acquisition 
threshold  criteria.  However,  to  encourage  VAN  participation  in  FACNET, 
DISA  made  the  decision  to  relax  the  certification  requirements. 

Certification  of  Questionable  Organizations  as  VANs.  As  a  result  of  relaxed 
certification  requirements,  DITCO  approved  organizations  with  questionable 
financial  resources  and  work  histories  to  perform  as  VANs.  Two  examples 
follow. 


0  One  company  had  a  moderate  Dun  &  Bradstreet  rating  suggesting 
closer  examination  was  appropriate,  had  no  prior  EDI  experience  (it  was  a 
manufacturer  of  navigational  equipment),  had  a  negative  net  worth,  and  had 
filed  for  voluntary  bankruptcy  within  1  year  prior  to  applying  to  become  a 
certified  VAN. 

0  Another  company  had  no  Dun  &  Bradstreet  rating,  provided  no  client 
references,  changed  company  names  twice,  and  had  fil^  bankruptcy  within 
3  years  prior  to  becoming  a  certified  VAN.  In  addition,  the  company  had  filed 
47  agency-level  protests  since  February  1990. 

Impact  of  Questionable  Financial  Resources.  Applicants  with  questionable 
financial  capabilities  should  not  have  been  certified  as  VANs.  Poor  financial 
data  at  the  start  of  a  contract  is  often  an  indicator  of  future  problems,  as  shown 
by  Inspector  General,  DoD,  Report  No.  96-105,  "Contract  Award  Decisions 
Resulting  in  Contract  Termination  for  Default."  The  report  states  that 
contracting  officers  awarded  24  contracts,  valued  at  $34.1  million,  to 
contractors  without  obtaining  adequate  information  to  support  determinations  of 
contractor  responsibility  or  without  adequately  addressing  adverse  contractor 
information  that  was  available  before  contract  award.  As  a  result,  $13.5  million 
of  unrecoverable  unliquidated  progress  payments  was  paid  to  defaulting 
contractors.  Additionally,  unquantified  administrative  costs  were  incurred  and 
operations  were  potentially  hindered.  DISA  certification  of  VANs  with 
questionable  financial  resources  could  result  in  similar  problems. 

DISA  incurred  additional  costs  by  certifying  15  VANs  with  questionable 
financial  resources,  despite  the  perception  that  the  DoD  affiliation  with  VANs 
was  a  no-cost  relationship.  Specifically,  DISA  incurred  administrative  and 
persoimel  costs  to  support  the  process  of  certifying  those  VANs  that  were 
questionable  financially. 

Improvements  Needed.  We  acknowledge  the  desire  DISA  had  to  obtain 
companies  to  function  as  VANs  and  to  encotirage  participation  during  the  initial 
development  and  implementation  of  FACNET.  However,  in  the  future,  DISA 
needs  to  improve  the  VAN  certification  process  by  enforcing  compliance  by 
DITCO  with  FAR  9. 104-1.  The  process  could  include  establishing  minimum 
financial  requirements  by  using,  for  example,  a  weighted  procedure  or  point 
system  for  evaluating  business  qualifications.  Though  &e  FAR  does  not 
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provide  specific  guidance  on  establishing  minimum  financial  requirements  when 
determining  contractor  financial  responsibility,  a  weighted  procedure  or  point 
system  can  be  useful  to  conduct  an  objective  and  fair  evaluation  of  business 
qualifications.  A  weighted  procedure  or  point  system  could  consist  of  assigning 
ratings  and  weights  to  each  category  being  evaluated.  For  example,  DITCO 
evaluates  a  prospective  VAN  based  on  its  customer  references,  financial 
institution  references.  Dun  &  Bradstreet  rating,  operational  business  plan,  and 
exclusion  from  the  vendor  debarment  list.  Relative  weights  could  be  given  to 
each  category  and  an  overall  score  or  rating  determined  for  each  prospective 
VAN.  DITCO  could  determine  an  acceptable  overall  score  or  rating  for 
considering  a  prospective  VAN  qualified  to  perform  electronic 
commerce/electronic  data  interchange  business  with  the  Government.  That 
procedure  would  make  the  review  more  objective  and  ensure  that  each  certified 
VAN  was  qualified. 


Compliance  With  the  VLA 


DISA  did  not  adequately  monitor  VANs  for  compliance  with  the  terms  and 
conditions  of  the  VLA.  Specifically,  DISA  did  not  perform  reviews  to  verify 
that  each  VAN: 

o  maintained  an  audit  trail  of  transactions, 

o  backed  up  all  data  to  allow  for  full  data  recovery  capabilities,  and 

o  had  an  internal  quality  monitoring  program  to  assure  the  maintenance 
of  reliable  communication  lines. 

The  VLA  requires  that  each  item  listed  above  exist  at  the  VAN.  VANs  must 
also  comply  with  additional  requirements  of  the  VLA  as,  detailed  in 
Appendix  C.  Although  our  review  did  not  include  checking  for  compliance 
wilh  the  additional  requirements  of  the  VLA,  the  remaining  VIA  requirements 
could  be  included  as  part  of  the  audit  trail  and  review  of  the  internal  quality 
monitoring  program  established  by  each  VAN. 

Audit  Trails.  The  VLA  requires  VANs  to  provide  the  Government  with  an 
EDI  mailbox  for  Government  use  in  monitoring  compliance  with  the  terms  and 
conditions  of  the  VLA  and  for  troubleshooting  and  testing.  The  VANs  must 
maintain  an  audit  trail  for  transactions  exchmged  through  the  DoD's  EDI 
mailbox  for  at  least  90  days.  The  audit  trail  should  include  the  date  and  time 
each  message  was  received  or  delivered.  The  mailbox  should  only  be  used  by 
the  DISA  DoD  technical  representative  who  is  responsible  for  administering  the 
VLA.  Despite  the  requirement  to  maintain  an  audit  trail,  the  DoD  technical 
representative  did  not  use  the  mailbox  to  monitor  compliance  with  the  VLA  and 
verify  that  the  VANs  maintained  an  audit  trail  of  transactions. 

The  Office  of  the  Deputy  Under  Secretary  of  Defense  (Acquisition  Reform) 
was  concerned  that  the  monitoring  of  the  mailbox  was  not  being  accomplished 
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and,  therefore,  hired  a  contractor  in  August  1995  to  develop  a  system  for 
performing  the  monitoring  function.  However,  as  of  February  1996,  the 
contractor  had  not  completed  the  development  of  the  system.  To  help  ensure 
compliance  with  the  VLA,  DISA  needs  to  monitor  the  mailbox  and  verify  that 
an  audit  trail  of  transactions  is  maintained  for  at  least  90  days. 

Without  assurance  that  VANs  are  monitoring  the  mailbox  and  maintaining  audit 
trails,  DoD  can  not  be  certain  that  VANs  are  in  compliance  with  the  terms  and 
conditions  of  the  VLA.  In  addition,  the  lack  of  monitoring  and  audit  trails  may 
prevent  DISA  from  identifying  and  correcting  problems  occurring  with 
electronic  transactions. 

Data  Backup  and  Recovery  Capabilities.  DISA  did  not  monitor  compliance 
with  the  VLA  for  backup  capabilities.  According  to  the  VLA,  the  VAN  must: 

o  back  up  all  data  it  processes  to  allow  for  full  data  recovery; 

o  provide  the  Government  with  the  capability  to  restore  EDI 
transactions;  and 

o  provide  the  Government  with  access  to,  and  use  of,  backup 
capabilities  after  disaster  notification  or  in  the  event  of  an  unplanned 
interruption. 

DISA  did  not  review  the  disaster  recovery  capabilities  of  the  VANs.  However, 
DISA  is  considering  adding  remote  testing  of  VAN  backup  recovery  capabilities 
as  part  of  tiie  new  VLA.  Until  DISA  adds  a  remote  testing  feature,  DoD  can 
not  be  certain  that  VANs  are  backing  up  data  as  required  or  that  DoD  will  be 
able  to  recover  transaction  information  in  the  event  of  a  disaster.  In  the  interim, 
DISA  needs  to  perform  periodic  reviews  to  ensure  that  each  VAN  has  a  disaster 
recovery  plan. 

Internal  Quality  Monitoring  Program.  DISA  did  not  verify  that  the  VANs 
had  internal  quality  monitoring  programs  as  required  by  the  VIA.  DISA  also 
did  not  require  the  VANs  to  provide  evidence  of  the  results  of  any  intemal 
quality  monitoring  performed.  According  to  the  VLA,  the  VAN  must  have  an 
intemal  quality  monitoring  program  that  assures  that  reliable  communication 
lines  are  maintained  to  enable  DoD  and  non-DoD  agencies  to  exchange 
electronic  transactions  using  the  EDI  mailbox  provided  by  the  VAN.  DISA 
personnel  stated  that  they  did  not  verify  whether  the  VANs  had  intemal  quality 
monitoring  programs  because  the  Office  of  the  Deputy  Under  Secretary  of 
Defense  (Acquisition  Reform)  did  not  provide  guidance  on  how  to  perform 
evaluations  of  intemal  quality  monitoring  programs.  However,  DISA  is 
responsible  for  verifying  compliance  with  the  VLA,  regardless  of  whether  DISA 
is  provided  guidance  from  the  Office  of  the  Deputy  Under  Secretary  of  Defense 
(Acquisition  Reform). 

Need  for  VLA  Compliance.  Audit  trails,  data  backup  and  recovery 
capabilities,  and  intemal  quality  monitoring  programs  are  measures  that  enable 
DoD  and  non-DoD  organizations  to  verify  when  VANs  are  responsible  for 
errors  and  omissions.  Without  use  of  complete  audit  trails  to  identify  problems. 
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DISA  must  rely  on  tedious  procedures  such  as  researching  trouble  tickets. 
Researching  trouble  tickets  involves  determining  the  causes  of  reported 
transaction  problems.  Trouble  tickets  document  problems  with  FACNET 
reported  by  VANs,  Network  Entry  Points,  trading  partners,  and  gateways. 
With  audit  trails  and  internal  quality  monitoring  programs,  DISA  should  be  able 
to  identify  and  correct  some  of  the  problems  occurring  with  electronic 
transactions,  such  as  late  and  lost  transactions,  that  are  resulting  in  trouble 
tickets.  Without  assurance  of  VAN  data  backup  and  recovery  capabilities, 
DISA  may  be  unable  to  pinpoint  where  problems  occurred  and  may  be  held 
liable  for  errors,  omissions,  or  nonperformance  by  the  VANs. 


Leniency  of  the  Current  VLA 


DISA  intended  for  the  current  VLA  to  be  lenient.  As  a  result,  some  controls 
were  not  built  into  the  VLA  to  protect  DoD  and  other  FACNET  participants. 
For  example,  the  current  VLA  does  not  include  detailed  communications  testing 
procedures  for  VANs  and  does  not  require  compliance  testing  prior  to 
certification  to  verify  that  VANs  can  send  actual  transactions  that  meet  the 
appropriate  standards.  In  addition,  the  current  VLA  does  not  include 
decertification  procedures  for  VANs  not  in  compliance  witii  the  terms  and 
conditions  of  the  VLA.  Also,  the  current  VLA  requires  that  DISA  annually 
reevaluate  the  terms  included  in  the  VLA;  however,  DISA  does  not  enforce  the 
requirement.  As  of  December  1995,  no  aimual  review  had  been  performed- 
According  to  the  DISA  DoD  technical  representative,  annual  reviews  were  not 
performed  because  DISA  weekly  meetings  and  focal  point  meetings  at  which 
VIA  issues  were  discussed  satisfied  the  intent  of  the  annual  review  requirement 
in  &e  VLA. 

Many  of  the  VANs  interviewed  voiced  concerns  that  the  VANs  certification 
process  was  too  lenient.  Specifically,  the  VANs  stated  that: 

0  DISA  was  not  performing  sufficiently  detailed  or  stringent 
communications  testing; 

0  DISA  was  certifying  VANs  that  could  not  pass  data  in  the  manner 
specified  in  the  VLA; 

0  DISA  was  not  testing  for  compliance  with  applicable  standards;  and 

0  DISA  was  certifying  companies  with  questionable  financial  and 
technical  capabilities. 

In  addition,  VANs  cited  concerns  that  the  current  certification  process  was  not 
adequate  to  ensure  that  VANs  already  certified  would  be  capable  of  handling  the 
transaction  workload  that  would  be  required  for  the  full  implementation  of 
FACNET. 
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Corrective  Actions  Taken  and  Planned 


DISA  is  taking  actions  to  address  some  of  the  issues  identified.  Specifically, 
DISA  is  in  the  process  of  revising  the  VLA  and  has  made  some  procedural 
chauQges  that  will  be  reflected  in  the  revised  VLA  that  may  improve  the  VAN 
certification  process.  Specifically,  DISA  transferred  the  responsibility  for 
communications  testing  of  VANs  to  the  Joint  Interoperability  Test  Command 
(JITC)  in  September  1995.  Our  review  of  the  JITC  test  plan  showed  that  JITC 
communications  testing  requirements  should  be  adequate  to  verify  that  VANs 
could  pass  data  in  the  maimer  specified  in  the  VLA.  However,  as  of  December 

1995,  JITC  had  not  begun  testing.  DISA  stated  that  JITC  will  not  begin 
communications  testing  until  the  revised  VLA  is  completed.  Under  the  revised 
VLA,  DISA  will  require  compliance  testing  prior  to  certification,  which  is  not 
required  under  the  current  VLA,  and  conduct  semiannual  reevaluations  of  the 
VLA.  Also,  DISA  plans  to  include  in  the  revised  VLA  circumstances  under 
which  VANs  can  be  decertified  and  the  procedures  for  decertifying  VANs  that 
are  not  meeting  the  requirements  of  the  VLA. 

We  acknowledge  the  improvements  that  the  procedural  changes  and  proposed 
VLA  will  bring  to  the  certification  process.  The  new  VLA  has  been  in  draft 
since  March  1995.  The  proposed  VLA  is  expected  to  be  implemented  by  July 

1996. 

The  proposed  changes  to  the  VLA,  however,  do  not  address  the  issues  we 
identified  regarding  improving  the  process  for  evaluating  financial  qualifications 
of  the  VANs,  monitoring  of  VANs  for  compliance  with  the  VLA,  and  enforcing 
the  VLA. 


Conclusion 


As  DoD  continues  to  implement  electronic  commerce,  DoD  must  be  able  to 
ensure  that  adequate  and  continuous  control  is  exercised  to  prevent  deficient 
services  and  quality.  To  encourage  VAN  participation,  DISA  relaxed  the 
requirements  in  the  VAN  certification  process  and  did  not  actively  monitor  and 
enforce  the  VLA  to  ensure  compliance  with  the  VLA  provisions. 
Consequently,  DoD  and  its  trading  partners  may  be  impacted  by  the  potential 
loss  of  business.  In  addition,  without  audit  trails,  data  backup  and  recovery 
capabilities,  and  internal  quality  monitoring  programs,  DoD  will  continue  to 
incur  unnecessary  costs  to  process  certifications  for  questionable  VANs  and  to 
trace  defective  transactions.  The  corrective  actions  taken  or  planned  will  not 
correct  the  problems  with  the  business  financial  review  of  VANs  and  with  the 
monitoring  and  enforcement  of  the  VLA.  Therefore,  DISA  needs  to  issue 
policy  requiring  DITCO  to  enforce  compliance  of  the  VANs  with  FAR  9.104- 
1 ,  and  to  monitor  VANs  for  compliance  with  the  VLA.  It  should  also  expedite 
the  completion  and  issuance  of  the  revised  VLA. 
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Recommendations,  Management  Comments,  and  Audit 
Response 


We  recommend  that  the  Director,  Defense  Information  Systems  ^ency: 

1.  Issue  policy  requiring  the  Defense  Information  Technology 
Contracting  OfHce  to  enforce  compliance  with  Federal  Acquisition 
Regulation  9.104-1  to  include  establish^g  a  i^stem  for  evaluating  business 
qualifications,  such  as  a  weighted  procedure  or  point  system. 

Defense  Information  Systems  Agency  Comments.  DISA  nonconcurred  with 
the  recommendation,  stating  that  the  DITCO  contracting  officer  exercised 
judgments  in  the  assessment  of  Value-Added  Networks'  qualifications  fully  in 
line  with  the  FAR.  DISA  maintains  that  the  assessment  was  reasonable  and  a 
preaward  survey  is  not  justified  because  the  VLA  is  a  no-cost  agreement.  The 
contracting  officer  verified  the  financial  references  and  the  lists  of  parties 
excluded  from  Federal  programs  and  indebted  to  the  Government.  DISA 
further  stated  that  because  the  capability  of  a  contractor  is  largely  subjective, 
judgments  generally  will  not  be  reviewed,  lacking  a  showing  of  fraud  or  bad 
faith. 

Audit  Response.  The  DISA  comments  are  not  responsive.  Relative  risk  exists 
in  relation  to  the  certification  of  financially  questionable  VANs,  even  though  the 
VLA  is  a  no-cost  agreement.  The  deadline  for  implementing  FACNET  is 
January  1,  2000.  To  encourage  participation  in  FACNET,  vendors  need  to  be 
certain  that  VANs  being  certified  are  financially  capable  and  will  be  in  business 
for  the  foreseeable  future. 

Though  the  determination  of  the  capability  of  contracts  is  largely  subjective,  the 
more  objective  the  evaluation,  such  as  though  a  weighted  procedure  or  point 
system,  the  less  likely  the  Government  will  be  subjected  to  loss  of  business  and 
lack  of  interest  by  vendors  in  doing  business  with  the  Government  through 
FACNET.  For  example,  for  cost-type  contracts,  the  contracting  officer 
evaluates,  assign  points,  and  selects  the  contractors  based  on  their  overall  score 
in  the  cost  or  price  and  technical  areas.  Regardless  of  whether  or  not  the  VLA 
is  a  no-cost  agreement,  the  key  to  implementing  the  new  concept  of  FACNET  is 
viability  of  the  VANs  and  the  credibility  they  present  on  the  behalf  of  the  DoD 
to  its  trading  partners. 

Using  a  weighted  procedure  or  point  system  would  make  the  review  more 
objective  and  ensure  prospective  VANs  were  considered  equally  and  fairly.  It 
would  also  eliminate  &e  contracting  officer  exercising  questionable  judgment  to 
determine  contractor  financial  responsibility.  Therefore,  we  request  that  the 
DISA  reconsider  its  position  on  this  recommendation  and  provide  comments  on 
the  final  report. 
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2.  Monitor  Value-Added  Networks  for  compliance  with  the  Value- 
Added  Network  License  Agreement  to  include: 

a.  Monitoring  the  test  mailbox  for  audit  trail  purposes. 

Defense  Information  Systems  Agency  Comments.  DISA  concurred  with  the 
recommendation,  stating  that  as  of  March  1996,  the  Office  of  the  Deputy  Under 
Secretary  of  Defense  (Acquisition  Reform/Electronic  Conunerce)  has  the 
capability  to  monitor  all  Government  transactions  being  transmitted  to  die 
various  VANs. 

b.  Performing  periodic  audits  to  ensure  that  each  Value-Added 
Network  has  a  disaster  recovery  plan. 

Defense  Information  Systems  Agency  Comments.  DISA  concurred  with  the 
reconunendation,  stating  that  the  Disaster  Recovery  Plan  will  be  recertified 
annually  as  part  of  the  implementation  of  the  new  VAN  licensing  agreement 
scheduled  to  begin  implementation  in  July  1996. 

c.  Verifying  existence  of  Value-Added  Networks  internal  quality 
monitoring  programs. 

Defense  Information  Systems  ^ency  Comments.  DISA  concurred  with  the 
recommendation,  stating  that  it  currently  monitors  networks  using  both  internal 
DISA  reporting  and  the  Office  of  the  Deputy  Under  Secretaty  of  Defense 
(Acquisition  Reform/Electronic  Commerce)  mailbox  system  identified  above. 

3.  Expedite  the  completion  and  issuance  of  the  revised  Value-Added 
Network  License  Agreement. 

Defense  Information  Systems  Agency  Comments.  DISA  concurred  with  the 
recommendation,  stating  that  it  received  formal  comments  to  the  new  VLA  from 
the  Office  of  the  Deputy  Under  Secretary  of  Defense  (Acquisition 
Reform/Electronic  Commerce)  on  April  30,  1996.  DISA  is  working  to  finalize 
that  document.  The  estimated  implementation  date  is  July  1996. 
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Value-Added  Network  Certification,  Management,  and 
Oversight  Review 


There  are  currently  25  Government-certified  VANs.  We  reviewed  the  current 
and  proposed  VAN  License  Agreements  (VLA),  including  the  technical  scope 
of  work  and  proposed  decertification  procedures.  We  identified  responsible 
personnel  and  held  discussions  with  DISA  and  DITCO  personnel  responsible  for 
VAN  certification,  management,  and  oversight.  We  reviewed  documents 
submitted  prior  to  certification  by  all  25  VANs  for  compliance  with 
FAR  9.104-1.  We  checked  for  standard  operating  procedures  on  certifying  and 
decertifying  VANs.  We  identified  personnel  responsible  for  providing  detailed 
written  test  plans  to  VANs.  We  obtained  and  reviewed  written  test  plans  where 
available  for  compliance  with  the  VLA.  Discussions  held  with  DISA  and  VAN 
personnel  covered  issues  pertaining  to  DoD  monitoring  of  the  test  mailbox, 
reviews  of  VAN  audit  trails,  data  backup  and  recovery,  and  the  existence  of  a 
disaster  recovery  plan.  Other  issues  discussed  included  the  VAN  internal 
quality  monitoring  program  existence,  communications  and  compliance  testing, 
and  the  VLA  annual  reviews. 


Audit  Period,  Standards,  and  Locations 


We  performed  this  program  audit  from  October  1995  through  February  1996  in 
accordance  with  auditing  standards  issued  by  the  Comptroller  General  of  the 
United  States,  as  implemented  by  the  Inspector  Genersd,  DoD.  We  included 
tests  of  management  controls  considered  necessary.  We  did  not  use  computer- 
processed  data  or  statistical  sampling  procedures  for  this  audit.  Appendix  E 
lists  the  organizations  we  visited  or  contacted. 
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Management  Control  Program 


DoD  Directive  5010.38,  "Internal  Management  Control  Program,"  April  14, 
1987,  requires  DoD  organizations  to  implement  a  comprehensive  system  of 
management  controls  that  provides  reasonable  assurance  that  programs  are 
operating  as  intended  and  to  evaluate  the  adequacy  of  the  controls. 

Scope  of  Review  of  Management  Control  Program.  We  reviewed  the 
adequacy  of  management  controls  over  VAN  certification,  management,  and 
oversight  at  the  sights  we  visited.  We  also  assessed  the  adequacy  of 
management's  self-evaluation  of  those  controls. 

Adequacy  of  Management  Controls.  We  identified  management  control 
wealmesses  as  defined  by  DoD  Directive  5010.38  relating  to  VAN  certification, 
management,  and  oversight.  Recommendation  2,  if  implemented,  will  establish 
controls  to  ensure  that  the  VAN  certification  process  is  adequate  and  that  VANs 
are  in  compliance  with  the  terms  and  conditions  of  the  VLA.  A  copy  of  the 
final  report  will  be  provided  to  the  senior  official  in  charge  of  management 
controls  for  the  electronic  commerce/electronic  data  interchange  program. 

Adequacy  of  Management's  Self  Evaluation.  DISA  officials  identified 
electronic  commerce/electronic  data  interchange  as  an  assessable  unit  in  a  self- 
evaluation  performed  in  August  1995  and  assigned  a  mediiun  level  of  risk. 
Because  we  did  not  review  the  entire  electronic  commerce/electronic  data 
interchange  area,  we  are  imable  to  determine  the  appropriate  level  of  risk. 
However,  VAN  certification,  management,  and  oversight  should  be  covered 
under  this  assessable  unit.  As  part  of  the  review  of  the  electronic 
commerce/electronic  data  interchange  area,  DISA  should  have  conducted  an 
evaluation  of  the  management  controls  applicable  to  VAN  certification, 
management,  and  oversi^t.  Because  DISA  did  not  conduct  an  evaluation  of 
the  management  controls  applicable  to  VAN  certification,  management,  and 
oversight,  DISA  did  not  identify  or  report  the  management  control  weaknesses 
identified  by  the  audit. 
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Other  Reviews 


The  Inspector  General,  DoD,  issued  three  reports  related  to  this  audit. 

Inspector  General,  DoD,  Report  No.  96-129,  "DoD  Implementation  of 
Electronic  Commerce  in  Contracting  for  Small  Purchases,"  was  issued  on 
May  24,  1996.  The  report  states  that  the  review  identified  a  series  of  issues 
involved  in  the  implementation  of  electronic  commerce  within  DoD.  The  issues 
identified  include:  realization  of  the  "single  face  to  industry"  concept;  adequacy 
of  the  transmission  of  data  by  the  DoD  FACNET  infrastructure;  implementation 
of  security  controls;  level  of  vendor  participation;  adequacy  of  management 
controls  for  FACNET  transactions;  and  adequate  development  of  FACNET 
implementation  plans.  The  Deputy  Under  Secretary  of  Defense  (Acquisition 
Reform)  and  DISA  are  aware  of  the  issues  and  are  implementing  corrective 
actions. 

Inspector  General,  DoD,  Report  No.  96-105,  "Contract  Award  Decisions 
Resulting  In  Contract  Termination  for  Default,"  was  issued  on  April  29,  1996. 
The  report  states  that  contracting  officers  at  Warner  Robins  Air  Logistics  Center 
awarded  24  contracts,  valued  at  $34.1  million,  to  contractors  without  obtaining 
adequate  information  to  support  determinations  of  contractor  responsibility  or 
without  adequately  addressing  adverse  contractor  information  that  was  available 
before  contract  award.  The  awards  resulted  in  a  $13.5  million  unrecoverable 
loss  to  the  Government  from  unliquidated  progress  payments  owed  by  the 
contractors  at  contract  termination.  The  report  recommends  that  the 
Commander,  Warner  Robins  Air  Logistics  Center,  establish  procedures  that 
contracting  officers  award  contracts  only  to  responsible  prospective  contractors 
and  that  determinations  of  responsibility  are  fully  supported  and  documented; 
establish  contractor  responsibility  determinations  as  an  assessable  unit  as  part  of 
the  management  control  program;  and  take  administrative  action  agaii^t 
personnel  involved  in  improper  contract  awards.  Management  concurred  with 
all  of  the  recommendations. 

Inspector  General,  DoD,  Report  No.  96-057,  "DoD  Use  of  Electronic  Bulletin 
Boards  in  Contracting,"  was  issued  on  January  8,  1996.  The  report  states  that 
the  use  of  bulletin  boards  by  DoD  procurement  offices  to  conduct  small 
purchase  transactions  was  not  a  major  impediment  to  FACNET  implementation. 
Bulletin  boards  served  as  an  interim  solution  that  enabled  procurement  offices  to 
conduct  electronic  commerce  until  FACNET  becomes  fully  operational. 
Procurement  officials  were  not  investing  significant  resources  to  establish  new 
bulletin  boards  or  to  upgrade  existing  capabilities,  and  they  were  committed  to 
phasing  out  their  use  of  bulletin  boards  when  FACNET  becomes  fully 
operational.  The  report  contained  no  recommendations. 
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Appendix  C.  Value-Added  Network  License 
Agreement  Requirements 

The  current  VLA  requires  that  the  VAN  transmit,  receive,  and  store  EDI 
messages  for  its  trading  partners.  Specifically,  the  VLA  requires  that  the  VAN: 

0  provide  DoD  with  an  EDI  mailbox  for  DoD  use  in  monitoring 
compliance  with  the  terms  and  conditions  of  the  VLA; 

0  maintain  an  audit  trail  for  transactions  exchanged  via  the  DoD 
mailbox  for  at  least  90  days; 

0  exchange  transactions  using  American  National  Standards  Institute 
Accredited  Stand^ds  Committee  X12  Standards; 

o  enable  interested  businesses  to  receive  and  send  Accredited  Standards 
Committee  X12  transaction  conventions; 

o  maintain  accessibility  for  exchange  of  transactions  to  and  from  the 
Government  Network  Entry  Points  24  hours  a  day,  7  days  a  week,  excluding 
8  hours  a  week  for  regularly  scheduled  maintenance; 

o  report  any  scheduled  and  unscheduled  breaks  in  service  under  the 
VLA  to  the  Government  in  a  pradent  manner; 

o  back  up  all  data  processed  in  such  a  way  that  full  data  recovery  is 
possible; 

o  maintain  an  internal  quality  monitoring  program  that  ensures  that 
reliable  communication  lines  are  maintained  to  enable  the  exchange  of  electronic 
transactions;  and 

0  provide  DoD  with  the  right  of  access  to  and  use  of  backup 
capabilities. 
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Appendix  D.  Organizations  Visited  or  Contacted 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition  and  Technology,  Washington,  DC 
Assistant  Secretary  of  Defense  (Conmand,  Control,  Communications,  and 
Intelligence),  Washington,  DC 


Other  Defense  Organizations 

Defense  Information  Systems  Agency,  Falls  Church,  VA 

Defense  Information  Systems  Agency,  C4  and  Intelligence  Programs,  Electronic 
Commerce/Electronic  Data  Interchange  Program  Management  Office, 

Falls  Church,  VA 

Defense  Information  Systems  Agency  Western  Hemisphere,  Fort  Richie,  MD 
Defense  Information  Systems  Agency  MegaCenter,  Columbus,  OH 
Defense  Information  Technology  Contracting  Office,  Scott  Air  Force  Base,  IL 
Joint  Interoperability  Test  Command,  Fort  Huachuca,  AZ 
Joint  Interoperability  and  Engineering  Organization,  Reston,  VA 
Defense  Logistics  Agency,  Alexandria,  VA 


Non-Goyermnent  Organizations 

Advanced  Communication  Systems,  North  Olmsted,  OH 

American  Logistics  Information  Corporation,  Diamond  Bar,  CA 

Advance  Logic  Resources,  Yaphank,  NY 

AT&T,  Philadelphia,  PA 

Complexity  Simplified  Incorporated,  Denver,  CO 

Computer  Network  Corporation,  Washington,  DC 

Datamatix,  Plymouth  Meeting,  PA 

Electronic  Data  Systems,  Herndon,  VA 

ELOCO,  New  Castle,  NH 

GAP  Instrument  Corporation,  Long  Island,  NY 

General  Electric  Information  Systems,  Rockville,  MD 

Harbinger  EDI  Services,  Atlanta,  GA 

Maple  Information  Systems,  Canada 

MCI  Telecommunications  Corporation,  Piscataway,  NJ 

Network  Iitformation  Services,  Newport  Beach,  CA 

Premenos  Corporation,  Concord,  CA 

Sidereal  Corporation,  Springfield,  VA 

Simplix,  Troy,  MI 
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Appendix  D.  Organizations  Visited  or  Contacted 


Non-Government  Organizations  (cont'd) 

Softshare  Information  Services,  Santa  Barbara,  CA 
Sprint  Government  Systems  Division,  Overland  Park,  KS 
Sterling  Software,  Dublin,  OH 
Technology  Management  Programs,  Carlsbad,  CA 
Total  Procmement  Services,  Novato,  CA 
TranSettlements,  Inc.,  Atlanta,  GA 
VAN  SAT,  Oklahoma  City,  OK 
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Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition  and  Technology 
Deputy  Under  Secretaiy  of  Defense  (Acquisition  Reform) 

Director,  Defense  Logistics  Studies  Information  Exchange 
Under  Secretary  of  Defense  (Comptroller) 

Deputy  Chief  Financial  Officer 
Deputy  Comptroller  (Program/Budget) 

Assistant  Secretaiy  of  Defense  (Command,  Control,  Communications,  and  Intelligence) 
Assistant  to  the  Secretary  of  Defense  (Public  Affairs) 


Department  of  the  Army 

Auditor  General,  Department  of  the  Army 


Department  of  the  Navy 

Assistant  Secretary  of  the  Navy  (Financial  Management  and  Comptroller) 
Auditor  General,  Department  of  the  Navy 

Department  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller) 
Auditor  General,  Department  of  the  Air  Force 


Other  Defense  Organizations 

Director,  Defense  Information  Systems  Agency 

Commander,  Defense  Information  Technology  Contracting  Office 
Director,  Defense  Contract  Audit  Agency 
Director,  Defense  Logistics  Agency 
Director,  National  Security  Agency 

Inspector  General,  National  Security  Agency 
Inspector  General,  Defense  Intelligence  Agency 
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Non-Defense  Federal  Organizations 

Office  of  Management  and  Budget 

Technical  Information  Center,  National  Security  and  International  Affairs  Division, 
General  Accounting  Office 

Chairman  and  ranking  minority  member  of  each  of  the  following  congressional 
committees  and  subcommittees: 

Senate  Committee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Services 
Senate  Committee  on  Governmental  Affairs 
House  Committee  on  Appropriations 

House  Subcommittee  on  National  Security,  Committee  on  Appropriations 
House  Committee  on  Government  Reform  and  Oversight 
House  Subcommittee  on  National  Security,  International  Affairs,  and  Criminal 
Justice,  Committee  on  Government  Reform  and  Oversight 
House  Committee  on  National  Security 
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DEFENSE  INFORMATION  SYSTEMS  AGENCY 


70iS.C0UfmCUS£AMD 
AHLMnOKVflGm  22204^91 


Inspector  General 


23  May  96 


MEMORANDUM  FOR  INSPECTOR  GENERAL,  DEPARTMENT  OP  DEFENSE 
ATTN:  DIRECTOR,  CONTRACT  MANAGEMENT 
DIRECTORATE 

SUBJECT:  Draft  Audit  Report  on  Certification  and 

Management  of  Value-Added  Networks 
(Project  No.  6CA-0019) 

Reference:  DODIG  Draft  Audit  Report,  subject  as  above, 

27  March  95 


1.  We  have  reviewed  the  subject  draft  report  and  concur  in 
part  with  the  finding  and  recommendations.  Our  management 
comments  are  enclosed  which  discuss  corrective  actions  to  be 
taken  or  that  have  already  been  completed. 

2.  The  point  of  contact  is  Ms.  Sandra  J.  Sinkavitch,  Audit 
Liaison.  If  you  have  questions  on  our  response.  Ns. 
Sinkavitch  can  be  reached  on  (703)607-6316. 


FOR  TEE  DIRECTOR: 


Enclosure  a/s 
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MANAGEMENT  COMMENTS  TO  COD  INSPECTOR  GENERAL  DRAFT  REPORT  ON 
CERTIFICATION  AND  MANAGEMENT  OF  VALUE-ADDED  NETWORKS 
PROJECT  NO.  6CA-0019 


Coasnanta  in  Reaponaa  to  tha  Finding t  DISA  concurs  in  part 
that  the  agency  had  not  established  an  adetjuate  process  for 
certifying  Value  Added  Networks  (VANS) .  The  main  point  of 
disagreement  is  the  DoDIG  assessment  in  regards  to  the 
Business  Qualification  Review.  The  DoDIG  report  cites  the 
Federal  Acquisition  Regulation  (FAR)  Part  9  as  the  criteria 
for  their  assessment.  We  agree  that  the  FAR  requires  an 
assessment  of  contractor  qualifications.  However,  the 
DoDIG' 8  evaluation  does  not  fully  describe  the  requirements 
of  the  PAR,  nor  does  the  finding  adequately  describe  the 
approach  taken  by  management.  Defense  Information  Technical 
Contracting  Office  (DITCO)  did  not  choose,  'not  to  strictly 
comply  with  the  policies  prescribed  in  FAR  Part  9"  as  stated 
in  the  draft  audit  report.  Rather,  the  contracting  officers 
utilized  the  flexibility  and  discretion  permitted  by  the  FAR 
language  to  make  a  reasonable  judgement.  The  extent  of  the 
information  necessary  to  make  a  responsibility  determination 
and  the  determination  itself  are  matters  within  the  broad 
discretion  of  the  contracting  officer.  Therefore,  the  issue 
is  not  one  of  compliance  with  FAR  Part  9,  as  DITOO  did 
perform  the  assessment  required.  Rather,  the  issue  is 
whether  the  assessment  was  reasonable  under  the 
circumstances.  The  facts  are  as  follows: 

1.  Electronic  Data  Interchange  (EDI)  VAN  License 
Agreement  is  a  no*co8t  agreement. 

2.  In  accordance  with  the  FAR  9.104-1,  all  potential 
EDI  VAN  Providers  were  determined  responsible  prior  to 
signing  the  EDI  VAN  License  Agreement.  The  process  for 
determining  contractor  responsibility  is: 

a.  EDI  VAN  Provider  must  submit  client  (4)  and 
financial  (1)  references. 

b.  All  references  were  verified  by  the 
Contracting  Officer. 
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c.  The  List  of  Parties  Excluded  from  Federal 
Procurement  or  Nonprocurement  Programs  and  the  List  of 
Contractors  Indebted  to  the  Government  are  verified. 

d.  Dun  &  Bradstreet  Report  (DfcB)  was  initially 
requested.  DITCO  also  uses  direct  contact  with  references 
and  others  as  necessary  to  determine  financial 
responsibility.  DITCO  did  not  limit  their  evaluation  to 
obtaining  a  D&B  report.  That  statement  in  the  audit  is  not 
factual.  The  assessment  was  based  on  relative  risk,  and  the 
initial  information  obtained  on  the  vendors  responsibility. 


A  Memo  for  Record  (MFR)  determining  contractor 
responsibility  is  placed  in  each  file  prior  to  certification 
that  the  above  information  is  current,  accurate,  and 
complete  to  the  best  of  the  Contracting  Officer's  knowledge. 
Contracting  Officer  signs  MFR. 

The  DoD  Inspector  General  report  cites  PAR  9.106-1  on 
preaward  survey  “when  information  on  hand  or  readily 
available  to  the  contracting  officer  is  not  sufficient  to 
make  a  determination  regarding  responsibility."  However, 

FAR  9.106-1  supports  DISA's  position  in  that  it  further 
states : 

the  contcsq>lated  contract  will  have  a  fixed  price 
at  or  below  the  siapllfled  acquisition  threshold,  the 
contracting  officer  should  not  request  a  prea%rard  survey 
unless  circumstances  jxistify  its  cost.* 

The  dollar  limitation  of  this  agreement  is  below  the 
Simplified  Acquisition  Threshold  (SAT)  (Currently  $100,000  or 
less)  as  the  contract  is  a  no-cost  agreement.  Therefore,  a 
detailed  contractor  responsibility  is  not  required  under  the 
SAT.  DITCO  cannot  justify  the  cost  of  conducting  a  preaward 
survey  for  a  no-cost  agreement. 

One  of  the  examples  cited  in  the  DoD  Inspector  General 
report  is  not  fully  explained.  The  issues  regarding  the  EDI 
VAN  Provider,  are  as  follows:  sent  information 

regarding  their  Chapter  11  bankruptcy  (Sep  93) .  Chapter  11 
bankruptcy  (reorganization)  does  not  automatically 
disqualify  a  coirpany  from  consideration.  The  facts 


26 


Defense  Information  Systems  Agency  Comments 


ascertained  by  the  contracting  officer  were  as  follows: 

1.  Reorganization  of  cotnpany  to  be  acccnplished  NLT 
Oct  94. 

2.  Received  approval  to  sign  agreement  from  DISA  HQ. 
Agreement  signed  Oct  94 

3.  emerged  from  Chapter  11  Ban]cruptcy  in  Oct  95. 


801OIARY  CONCLUSION;  DISA' 8  position  previously  stated  by 
GAO  is  that  'Because  the  determination  that  an  offeror  is 
capable  of  performing  a  contract  is  largely  sobjective,  such 
judgments  generally  will  not  be  reviewed  absent  a  showing  of 
fraud  or  bad  faith*  (Reference:  Color  Dyneunics,  B-250398, 
1993  West  law  17602 (C.G.) (Jan  22.  1993) .  The  DITCO 
contracting  officer  exercised  judgements,  in  the  assessment 
of  this  agency  fully  in  line  with  the  relative  risk  these 
contracts  posed  to  the  Government  in  accordazice  with  the 
provisions  of  FAR.  Therefore.  %fe  cannot  agree  with  the  DoD 
Inspector  General's  assessment. 

VAN  Testing:  DISA  does  concur  with  the  DoD  Inspector 
General's  assessment  regarding  VAN  testing.  Specifics  are 
provided  in  response  to  the  recoanendations. 


RXCOKKSNDXTIONS : 

We  recommend  that  the  Director,  Defense  Infonation  Systems 
Agency: 

1.  Issue  policy  requiring  DITCO  to  enforce  con^liance  with 
FAR  9.104-1  to  include  establishing  a  system  for  evaluating 
business  qualifications  such  as  a  weighted  average  procedure 
or  point  system. 

DISA  Response:  Non- Concur,  DISA's  current  procedures  are  in 
accord  with  FAR  requirements  as  explained  in  our  response  to 
the  finding.  Therefore,  DISA  sees  no  need  to  revise  current 
procedures  to  determine  contractor  responsibility  lAW  FAR 
9.1  and  DFARS  Subpart  209.1. 
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2.  Monitor  Value-Added  Networks  for  courpliemce  with  the 
Value-Added  Network  License  Agreement  to  include; 

a.  monitoring  the  test  mailbox  for  audit  trail 
purposes, 

DISA  Response:  Concur,  as  of  March  96,  the  DUSD(AR/EC)  has 
the  capability  to  monitor  all  Government  transactions  being 
transmitted  to  the  various  VANs.  Additionally,  DUSD{AR/EC) 
also  has  a  mailbox  (just  like  any  vendor)  with  each  VAN. 
This  allows  the  Government  to  monitor  Government 
transactions  being  transmitted  to  the  VANs  as  well  as  being 
atble  to  monitor  transaction  processing  compliance  from  a 
vendor  perspective.  This  capability  allows  the  Government 
to  verify  transaction  processing  and  retain  an  audit  trail. 

b.  performing  periodic  audits  to  ensure  that  each 
Value-Added  Netwrk  has  a  disaster  recovery  plan,  and 

DISA  Response:  Concur,  the  Disaster  Recovery  Plan  will  be 
recertified  annually  as  part  of  the  Implementation  of  the 
new  VAN  licencing  agreement  scheduled  to  begin 
implementation  July  1996. 

c.  verifying  existence  of  Value-Added  Networks 
internal  quality  monitoring  programs. 

DISA  Response:  Concur,  Ne  currently  monitor  networks 
utilizing  both  internal  DISA  reporting  and  the  DUSD(AR/EC) 
mailbox  system  identified  in  response  to  (a)  above.  These 
two  mechanisms  allow  us  to  provide  daily  management  of  the 
supporting  communications  infrastructure. 

3.  Expedite  the  completion  and  issuance  of  the  revised 
Value  Added  Network  License  Agreement. 

DISA  Response:  Concur,  He  received  formal  comments  to  the 
new  VAN  license  agreement  from  DUSD(AR/EC}  30  April  1996. 
DISA  is  working  to  finalize  this  document.  Estimated 
implementation  date  is  July  1996. 
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